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a study was undertaken to Identify those priorities, specifically, it 
looked for the qualities of instruction that medical and pharmacy 

students consider most important in their l^a'^^^?" . Li^Si«5L?« 
instructional characteristics was developed and divided randomly into 
tuo lists, which were then distributed randomlytcincoaing students 
of the 1983 class at the Oni-versity of RrixonalWITege of Medicine 
and the senior class- of the College of Pharmacy, for ranking. 
Responses were received from 145 students. Analysis of the survey 
responses shows no significant difference in attitudes in the two 
schools, although medical student responses were in general more 
hoiogenetus and a few salient instructional qualities were ranked 
very differently by the two groups. Like groups previously surveyed, 
these students consider clarity and organization of content and 
presentation to be of great iapottance: unlike ether groups they 
consider instructor knowledge of lesser importance. It is suggested 
that this may be due to the survey instrument or to the substantial 
notf^ teacher irformatlcn resources available to these professional 
students. Further research on this and related issues is recommended. 
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Abstract 



College and graduate students are frequently called upon to 
evaluate their courses and instructors. The evaluation instruments 
are derived from general principles of learning, instructional 
guidelines, or teaching heurlistics. The'Vatings are made by students 
within a framework of priorities regarding certain qualities of the 
f instruction they received. Instructional evaluation has not always 
been closely responsive to such student priorities. The first step 
in a better resptinse is to identjfy such student priorities. 

This brief study sought to identify those qualities which medical 
and pharmacy students consider to be most important to their learnrng. 
A survey methodology involving student ratings of 39 instructional 
qualities was used. The results, quite consistent across student 
groups, clearly identified three qualities as most ^^sential to their 

H 

learning m 
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College, graduate and professional students are frequently called upon 
to evaluate thei r-courses and Instructors. This practice Is'lntcnded to 
yield Information useful In monitoring Instructional quajlty, documenting 
teaching contribution, and providing suggestions for Instructional Improve- 
ment. The evaluation Instruments used are usually comprised of a general 
• set ofMeachIng heur'mics, principles of learning, or instructional 
guidelines thought to affect learning combined with some type of rating 
" ^ scale. . . . 

The Items corerron to Instructional evaluation instruments, whether . 
* derived intuitively or empirically, have as their base some mode l^^^^p-^'fchetrta 

of .effective Instruction. Feldman (1976) 4n his synthesl-S and meta-^analysi s 
of/ some 70 studies o" teaching qualities report* many studies which describe 
a "good teacher" or which Identify behaviors thought, to discriminate 
between the best and the worst Instructors. 

Irby (1978 a & b)- has delineated four dimensions or qual i tl es which 
appear regularly In studies as b^ing hrlghly related to effective classroom 
^ teaching and which also seem to be related to effective clinical teaching. 

These Include: (1) organization/clarity; ^2) enthusiasm/stimulation; (3) 
instructor knowledge; and {k) group instruVtional skill. In his proposed 
self-assessment inventory for clinical and classroom teachers In medicine 
he has added the dimensions of (5) rapport; (6) c.llnlcal supervision; 
(7) clinical competence; and (8) professional characteristics (1978 a)." 
The fiVst five character! sties general ly correspond to the dimensions which- 
Feldman examined /al though they are broader categories thar/Feldman ' s . 
The practice of students ra444ag instruction rests upon the assumption that 
students h-ave formulated a structure or set of criteria which define 
effective instruction for them. 

« / - - II ' • I • 
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An assumption In mnny professional school', is that instruction 
evaluation instruments derived for use in hlqher education in general 

♦ 

will have direct application to the evaluation of instruction in the 
professions. In exploring this assumption this sttjdy sought to identify 
the qualities of instruction considered by medical and pha^rmacy" students 
to be important to their learning. It was expected that a subset of • ^ 
essential qualities would emerge from the more important qualities identified 
and that the" compos 1 1 ion of the subset would be consistent across student 
populations. The existence of such a subset would support the assumption 
that students do have a schema for qualities of instruction which they 
consider essential to learning. 

Method 

A questionnaire consisting of 39 items, derived to be consistant wi th 
Irby's and Feldman's dimensions, was randomly divided into two scales (A and 
B) of 20 items and 19 Items respectively in order to reduce the time required 
to complete the form. The scales were then randomly distributed to all 
members of tb^jj^mlng class of 1983 of the University of A_rizona College of 
Medicine and the senior class of the College of Pharmacy during their first 
week in session. A total of 92 responses were obtained from medicine and 
53 from pharmacy, approximately equally divided between scales.- 

The questionnaire was of a 3 ?oint structured-response design on whicji 
the respondent identified each behavior or quality listed as being "Essentlar\ 
"Desirable", or "Of Littie or No Importance" to his/her learning. In analyzing 
the data a numerical value was assigned to each o*f the thre°e classifications 
with Essential receiving a value of 3", Desirable a 2, and Of Little Importance 
a 1. The mean and standard deviation for each Item were obtained. 
Discriminant analysis was used to see if there was a statistically significant 
difference between the medical and pharmacy ^students' responses. Spearman 
Rank-Order Correlation (rho) was used to determine whether and to what degree 
the rankings. by the two groups were related. 




Resul ts 

Items appeared to cluster roughly Into four groups. The Item clusters 
at the extreme ends of the range of meani wore clearly separated froin the 
two large middle clusters, lliat Is, both qroups of students seemed to 
consider three I terns essential, and both groups had considerable concensus 
on these Items (means ranged between 2.?^ and 2.88 with standard deviations 
ranging between .32 and .^5). These Items were: clearly communicates what 
Is expected to be learned, answers students' questions carefully and precisely, 
and emphasizes what is Important. These rVprosented Irby's qualities of 
organization/clarity and instruct ional ski 1 1 . 

The second clustering of responses (see attached list of items rank 
ordered) had a broader range for both means (2.25 to 2.62) ai>d standard 
deviations {.kS to .65). Among the top 13 Items are represented four of 
the five. di\nen4lons I rby denotes as being important on a general level to 
most students. Of the top 13 i tents' listed, 9 are common to both student 
populations. The Items In addition to those already identified were: 
reMates facts to form concepts,* tests/the important course material, corrects 
Student mi stakes wi thout belittling them, reviews essential material, paces 
presentation to student rate of comprehension, and assigns grades fair.l^^. 
The top twelve ranked items do not include several qual i ties Ranked highly 
In the studies reported by Feldman. Though low In thjjs 5.tudy's rankings, 
qualities like Instructor preparedness, instructor ability to st:mulate 
interest, and Instructor knowledge of the subject ranked third, fourth, 
and fifth in Feldmans' -resul ts. 

Discriminant analysis showed no sigftificani difference between the - 
medical and pharmacy student responses. The medical student population 
appeared to be a bit more homogeneous (SD range .363 - .705) than the 
pharmacy student population (.320 - .77'*) in their responses.. Spearman rho 
yielded a significant correlation of .81, jJ_<.Ol between the rankings of 
the medical and pharmacy students. 




The following items were. ranked high by medical students and low by 

pharmacy students: . • ' 

Med Rank 3 Relates new material to previous learning Pharni Rank 19 
Med Rank k Shows enthusiasm about the subject Pharm Rank 23 
Med Rank 7 Discusses points of view other, than his/her own Pharm Rank 28 
Fi.ve of the six'items wi th, the lowest ranking were identical. The 
qualities of' encouraging active participation in discussion, utilizing aud^lo 
visual resources' effectively, directing students to useful literature, bring 
self critical and not appearing arrogant received the lowest ratings. 

• / 

Dis cussion and Conclusion r . . • 

The results obtained in 'this study .seem to support* the expectation that 
professional (at least medical and pharmacy) students do have a schema for 
qualities of ife§trucfton"whiciS,are important to learning. Whether they use ^ 
this schema whe^^^tually evaluating i hstruct i<^n is a subject for ^another 
study. The components of this schema aV still not fully n<rr def i.ni ti vel^y 
delineated, nor has the question ef whether professional students xomprVse 
l» population unique from any tJther^ group of students been answered. It 
does appear that these student groups consider cla-rity and organi zat ion-both 
of content and presejjtat ion to be of hi'gh importancfe.. In this they are much 
. like other college students who have been studied. -The students in this 
study differed from other 'students . however, in ranking instructor knowledge 
as being of much less importance. It may be that this is a result of the 
instrument .,,»d. Since there were as many items from this dimension as 
•from-the dimensions of cjarity or enthusiasm, the first explanation seems 
less likely than that rredlcal students assume that their faculty is knowledgcabl< 
They may further assume that there are ample resources available for 
obtaining factuvil information other than their instruetor. It Will be 
interesting to see if this ranking will change as students move" into their 
clinical training. 

There seem to be more questions raised by this study than were answered. 
For example: Would other student populations rank the importance of qualities 
.in a different way than do these student's? Is there less variance among 
medical and pharmacy students than among law students 6r educational psychology 
graduate students? Are, the qualities freshman students deem essential 



different from those essential to seniors? Do students and faculty agree 
on the essential qualities of go^d instruction? Do students rate Instruction 
according to their schema for "good teachim,"? Further research into these 
questions is currently being conducted. We may tentatively conclude from 
this study that medical and pharmacy students consider clearly defined 
expectations of student learning, careful answerinq of students' questions 
and emphasis of important concepts are cs>>i'ntlal Inbturctor behaviors for ' 
effective learning. 
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Rank Crd.cring of Instruction Qualities 



Medical Students 
Rank Mean SD ltetn# 



. Pharmacy Students 



1. 
2. 
3. 



5.5 
5.5 

7. 
10. 
10. 
10. 

• 10. 

9 



15. 

15. 

15. 

17.5 

17.5 

19. 

20.5 

20.5 



2.761 
2.739 



.363 
.^31 



11 
7 

n 



Clearjy convnunicates what is expected to be learned 
Answers student questions carefully and precisely 
Emphasizes what is important 



2.587 .S'*! »17 Relates new material to previous learnings 

2.565 .501 21 Shows enthusiasm about the subject 

2.565 .S'tA Ik Tests the important course material 

2.5J43 .5'«6 29 Relates facts to form concepts 

2.522 .505 \k Discuses points of view other than his/her own 

2.522 .505 3 Corrects student mistakes without "bel ittl li^g them 

2.522 .5^7 18 Reviews essential material 

2.522 .5^7 38 Paces presentation to student rate of comprehension' 

2.522 .5'»7 36 Assig/^s grades fairly 

2.W .535 6 Gears l-nstruction to student's level df readiness 



2.W 
2.435 

2. 413 
2.391 
2.3^»8 
2.3^8 



.5i|i» 32 Wllllftgly remains accessible to students 

r5A4 37 . Summarfzes major points 

'.620 Clarifies confusing examination ques tons 

.5^11 28 Willingly explains further 

.^k\ 1 Seems to enjoy teaching 

.537 26 Stimulates student Interest in the subject 

.^^82 .8 Is open-minded' 

.526 23 Quickly grasps what students are asking telling 



Rank 


Mean 




1 


2.8&9 . 


.320 


3 


2.731 


.452 


2 


2.8A6 


.368 


19.5 


' 2.3'*6 


.'485 


2k 


2.2&9 


.447 


5 


2.593 


.501 


12' 5 


2.'48l . 


.509 






.653 


in 


2 500 


.510 


L 


2.615 


.496 


8 


^2.556 




6 5 


2^577 


.578 


6.5 


2.577 


.643 


22 


2.296 


.609 


12.5 


2.481 


.580 




2.519 


.509 


16.5 


2.407 


.501 


,^5 


2.423 


.504 


2f 


2.296 


.465 


10.5 


2.500 


.510 


29 


2.111 


.506 



10 
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Medical Students 



Pharmacy Students 



Rank 


Mean 


SD 


1 tern fr 


22 ^ 


2. ■?26 


. 560 


39 


22 ^ 


2. '526 


.519 


2 


7k ^ 


2 2?9 


. 565 


35 


7U ? 
£*i • ^ 


2.2^9 


. 705 


16 


26 


2. 196 


.582 


33 


27 


2 n't 


. 529 


15 


28 ^ 


2. 152 


. 556 


13 


28 ^ 


2. 152 


.595 

■ ^ ✓ ✓ 


20 


^u. 


2.087 


.590 


. 10 


J ' • 


2 0^3 


.556 


5 


"^2 


2 022 


.577 


27 




1 978 


.577 


25 


XX ^ 


1 /978 


.683 


12 


•35. 


1.957 


.556 


'31 


36. 


1.935 


.574 


19 


37.5 


t.804 


.500 


30 


37.5 


1.804" 


.6ia 




37.5 


,1.761 


.639 


9 



Encourages a cHmate of mutual respect 
Listens attentTvely 

Has an interesting style of presentation 

Takes responsibility for own actions 

Reveals kno\*Jedge in his/her discipline 

Provides support and encourdqement to students 

Quest ions'* students to elicit underlying reasoning 

Discusses eurrent developments in his/her spedialty 

Shows personal interest in students 

Giving students positive rei nforcemer*^ for good 

fti2i5AV!^ite'fiRfCl?^S^§^^ 

Seems to have self confidence 

Uses class handouts effectively 

Encourages active participation in discussion 

4Jtili2es audio visual resources effectively 

Directs stuiJ'e'nts to useful literature in the field 

Is self crltlca.1 

Does not appear to be arrogant 



Rank 


nean 




SD 




2.259 






2.i462 




508 


33 


2.000 




19.5 


2.3^6 




689 


16.5 


2.1«07 


.572 


18 


2.385 


.i«96 


36 


1.885 


« 


516 


31 


2.038 




77'* 


26.5 


2.231 


t 


51A 


26.5 


2.231 


587 


Ik 


2.259 




j656 


yi 


2.037 




I587 


30 


2.077 






39 


1.815 




3^.5 


1.962 


.662 


37 


1.852 




.362 


38 




.732 


3^».5 


1.962 




.720 



Name 



I NSTRUCT I QNAL QUAl \ T 1 tS»_ SCAU„:A 

This instrumciiU i intended ti> coHoct y'ln in-llvidu.l r,n irvjs (;f v,jriou3 

instructional qujlitics which, hctsaJ p,ist".vi , r i. iKt', mat- o a s i yni f i c.iiit 

contribution to your Icarnlny. 
« 

The method used here required that you dctcr:iunc the i nipoj'tance of each in- 
structional quality in relation to the other iju.ililies listed. This nieans that 
some of the qualities you select as niore imp«Mtnnt than others, m-iy be only 
slightly more -«iir.port ant . 

IMease rate the importance, lo yon In y uir hMininj, <»f ench q.olity listed. 
Using the scalo provided below assign a rot in.; IrtU'r to c.ich item. 

E - a quality which is ESSENTIAL to, roc in i«y loarninq * 

D « a quality which is generally OESIRAuLf: t^. mc in my learning 

L » a quality which is of LlTTlE or no inpm tanco to in my Icnrnlr.g 



I. 


Seems LO enjoy toadii'i^j 


• 


Liblori'i allt'iuively 


3- 


Ccjrrects stmknt mi^t^■.-^ .nthoul Lt:littlintj the" 


U. 


1 sc 1 f cri ♦ icn 1 


5. 


Giving i.tudcnls po'jilivc [o i nforce^ient for qoud con- 




tributions 1 n per forr'-Tncc 


6. 


Gears i n'itrutrKl.on^to stn-lcnt's level of readiness 


7. - 


Answers student qui-stiuiT^ c.. ireful 1 y and precisely 


3. 


Is opcn-ni ndt;d 


'i. 


Does not .ippt-at t<> 1h- .iri . ..iiit 


'\ 


Shows pervonal interest in studcnii. 




E|mphasi2cs -what is import.tnt 


12. 


ijses class handouts effectively 


13. 


(Jiuestions students to elicit underlying reasoning 


111. 


Discusses points of view other than his/her own 


15.- 


Provides support and encouragement to students 


16. 


Takes responsibility for own .ictions 


17. 


Relates new material to previous learnings 


18. 


Reviews essential material 


'5. 


Utilizes audio visual resources effectively 


20. 


Discusses current develop'^umts in his/her specialty 



INSTRUCTIONAL QUALITIES SCALEjiP. 



This instrument is iPrrm4^ed to collect your individual ratings of various 
Instructional qualities which, ^based on past ri. ncc, mat-re a significp-t 
contribution to your learning. 

Tlv iwthod u'jed here required that you doKmuno the inportance of each in- 
structional quality in relation to the other quilities listed. This means thaf 
soinc or the qualities you select as more import.int than others may be only 
s I i ght Iv i^wre important. 

Please rate the ia.portancc, to you in yowr l-.imii.y. of each .quality listed. 
Using the scale provided below assign a ratin.j letter lo c.ich item. 

Z - quality which is ESSENTIAL to me in V '^'^•^^^^'i . 

D - a quality which is generally OESiaACU to »k in ..,r-^«ir. - ^ 

L - a quality which is-of LITTLE or no importance to me in my learrnny 

2 1. Shows enthusiasm about llic '.ubjcct 

22. Clearly compunicates whar i- expected to be iearned 

23. Quickly grasps what students are asking or telling 
Zk. Tests the important course material 

2*5. Seems to have self confidence 

2G. Stimulates student interest in the subject 

^•7. Recognizes own limit.iiinn^ 

.:3. Ui 1 Unqlv explains furin.-i 

29. Relates facts to •for.m concepts - 

-vo .Directs students to useful literature in the field 

. Encourages active participation in discussion 

^2. Willingly rcmain» accessible to students 

.y.3. Reveals knowledge in his/her discipline 

Clarifies confusing examin.ition questions 

j^5. Has an Interesting styl«' of presentation 

.rb. Assigns grades fairly 

J7. Summarizes major points 

,8. Paces presentation to student rate of comprehension 
Encourages axlimate of mutual respect 
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